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Abstract 

Image registration is an important element in data pro- 
cessing for remote sensing with many applications and a 
wide range of solutions. Despite considerable investigation 
the field has not settled on a definitive solution for most ap- 
plications and a number of questions remain open. This 
article looks at selected research issues by surveying the ex- 
perience of operational satellite teams, application-specific 
requirements for Earth science, and our experiments in the 
evaluation of image registration algorithms with emphasis 
on the comparison of algorithms for subpixel accuracy. We 
conclude that remote sensing applications put particular 
demands on image registration algorithms to take into ac- 
count domain-specific knowledge of geometric transforma- 
tions and image content. 


1. Introduction 

Image registration, the alignment of images and other 
data for comparison and fusion, is an important element in 
data processing for remote sensing. It finds many applica- 
tions, including but not limited to the operational production 
of satellite image products, validation and precision correc- 
tion of satellite navigation data, radiometric validation of 
new sensors, data fusion for classification and thematic map 
production, temporal image comparison for change detec- 
tion, atmospheric velocity computations, and land use anal- 
ysis and planning. 

This varied set of application needs, as well as similar 
needs in allied fields like medical imaging, have driven the 
development of a large number of approaches for image reg- 
istration (see Brown [5], Fonseca and Manjunath [12] and 
Zitova and Flusser [35]). In an editorial on medical image 
registration Pluirn and Fitzpatrick [22] note that since 1988 
the number of related articles per year - has grown about 10- 
fold, from approximately 15 to over 140. 



Figure 1. Mulitsensor registration of ETM+ at 30m (surround) to 
IKONOS at lm (inset) 

Image registration for remote sensing, particularly in 
support of operational production of imagery products, can 
be quite automatic, efficient and effective. However, many 
end users still depend on labor-intensive manual selection 
of control points (CPs) or, to quote one researcher, “reg- 
istration by graduate student,” with times cited from a few 
hours to 4-5 days for registering a single image [8, 13]. This 
article explores this disparity in the use of advanced tech- 
niques, discusses selected issues in evaluating automatic 
methods and fostering their greater adoption, and outlines 
the research program in image registration conducted by the 
image registration working group at NASA Goddard Space 
Flight Center (GSFC). 

In this article we describe image registration as a process 
that aligns one image to another of the same area taken at 
the same time or later, and with the same sensor or a dif- 
ferent one. An image registration method can be defined 


by three main steps: (1) extraction of features to be used to 
match the imagery; (2) feature matching strategy and met- 
rics, commonly the optimization of a similarity measure be- 
tween images; and (3) resampling or indexing of the data to 
a common coordinate system [5]. The transformations al- 
lowed for alignment can vary, from translation only in x and 
y, to rotation, scale and translation (RST), to fully-affine, to 
homographic, or to a more complex model [32, 26]. Many 
current systems use two levels of transformation as they 
match CPs as small image regions (chips) using only trans- 
lation or a low order transformation, and then use the CPs to 
compute a more complex transformation. Some algorithms 
use Digital Elevation Models (DEMs) to correct for parallax 
caused by terrain relief. 

2. Application requirements for image regis- 
tration 

In part, the wide variation in image registration tech- 
niques in remote sensing is a result of the wide variation in 
applications. Fonseca and Manjunath [12] noted this, pro- 
posed a rule-based system that would automatically select 
an appropriate method for an application and reviewed the 
literature to characterize the available methods to that end. 
Our own work [16] supports this goal. In particular, we 
have implemented a number of methods to evaluate them in 
a consistent framework and to develop an online toolbox of 
techniques for end users. 

This program also requires understanding application 
needs to get an end-to-end view of how image registration 
fits into overall data processing. The literature on image 
registration is dominated by articles advancing or propos- 
ing a new technique, with far fewer articles that document 
user requirements or survey current practice. In this section 
we look at issues in user requirements, dividing users into 
two groups: operational groups that use image registration 
to produce image products, and Earth scientists (and others) 
who use the products. 

2.1. Operational system requirements and practices 

Operational satellite science teams that support remote 
sensing sensors rely on image registration for a number of 
purposes, including updating a geometric model that relates 
a sensor to the navigation data (position and orientation) 
for a satellite; updating the navigation data; image by im- 
age geolocation; sensor-to-sensor and intra-sensor calibra- 
tion; and validation of model and data quality. Geolocat- 
ing a satellite image relative to a ground coordinate system 
can be done in two primary ways: systematic correction us- 
ing satellite navigation and calibration data, and precision 
correction through registration of image features to previ- 
ously located ground features. Precision correction can be 
done indirectly, by updating the camera models used for 


systematic correction, or directly by relating a new image 
to a ground coordinate system. 

The teams have an end-to-end view of data processing 
requirements for image production. From an early point in 
sensor design they develop specifications that include im- 
age registration reliability and accuracy requirements, and 
they typically implement in-house software to those re- 
quirements. Their knowledge of the satellite and sensor 
enables them to model the radiometric and geometric trans- 
formations between images so the image registration algo- 
rithms can be tuned and integrate satellite navigation data. 
Below we survey six representative operational systems to 
look for characteristics and commonalities. 

ASTER is a 14-band multispectral imager launched in 
1999 on the Terra (EOS-AM1) satellite. It has three sepa- 
rate image subsystems with individual telescopes that sam- 
ple at 15m, 30m and 90m. Iwasaki and Fujisada 2005 
[14] describe the image registration system. Image reg- 
istration was intended to improve geolocation of imagery 
to within 100m (nadir), adjust intratelescope band-to-band 
position to within 0.2 pixels and adjust intertelescope posi- 
tion to within 0.3 pixels. Registration is done to a database 
of about 300-600 CPs which were mapped on topographic 
maps to to about 2-4 meters in x, y and z. The similar- 
ity measure is normalized correlation with transformation 
limited to translation. Subpixel estimation is calculated by 
fitting a second-order polynomial to the correlation values. 
Matches are rejected for correlation less than 0.7 or when 
clouds are detected. Validation of satellite performance was 
done in a retrospective study and the authors cite accuracies 
of 50m, 0.2 pixels and 0.1 pixels, respectively. Problems 
were noted with parallax errors in high-elevation areas due 
to time delays between bands, with cross-track performance 
due to satellite jitter, failures by the cloud detection algo- 
rithm, and cross-band matching in scenes with considerable 
vegetation. 

GOES I-M is a series of weather satellites launched 
from 1994 to 2001 with seven bands that sample at 1km, 
4km and 8km. Madani, Carr and Shoser 2004 [20] describe 
the AutoLandmark system for image registration. GOES 
geolocation requirements vary from night to day, from 4km 
to 10km, to accommodate solar heating. Registration is 
done to a database of coastlines represented in vector for- 
mat which is used to avoid radiometric issues in correlation. 
A landmark of 24x24 or 96x96 pixels is extracted from a 
sensed image and edges detected using the Sobel operator. 
This is matched to a binary image rasterized from the coast- 
line vector database, with a correlation measure of the sum 
of the edge strengths and transformations limited to transla- 
tion. Cloud areas are masked out. Bisection search is used 
to find subpixel maximums. Matches are rejected when a 
fuzzy logic quality metric (QM) falls below a threshold of 
0.9. The QM takes into account the quality of the edge cor- 



Instrument 

Satellite 

Resolution 

Similarity 

Subpixel approach 

ASTER 

Terra 

15-90m 

Correlation to DEM corrected CPs 

Fit to correlation surface 

GOES 

GOES I-M 

l-8km 

Correlation to vector coastline database 

Bi-section search 

MISR 

Terra 

275m 

Correlation to DEM corrected CPs 

Least squares optimization 

MODIS 

Terra 

250m - lkm 

Correlation to DEM corrected CPs 

Fixed regular sampling 

HRS 

SPOT 

2.5 m 

Correlation to DEM corrected data 

Not described 

ETM+ 

Landsat-7 

15-60m 

Correlation to arid region CPs* 

Fit to correlation surface* 

VEGETATION 

SPOT 

lkm 

Correlation to DEM corrected CPs 

Not described 


Table 1. Operational Image Registration Systems. * ETM+ information in table refers to band-to-band registration 


relation, the fraction of cloud contamination, and the con- 
trast and possibly total illumination in the image. In the 
infrared bands the daily warming/cooling cycle means land 
and water can reverse in radiation, causing coastal edges to 
invert, migrate and even disappear. 

MISR is a second instrument on the Terra satellite. Jo- 
vanovic et al. 2002 [15] describe the geometric correction 
system. It has 9 pushbroom cameras with four bands each, 
all sampling at 275m, intended to image atmospheric ele- 
ments from multiple angles for climate analysis. This repre- 
sents a particular challenge for geolocation of the multiple 
cameras with the objective of achieving better than 140m 
uncertainty in absolute geolocation at nadir and between 
cameras. Registration has been done to a database of 120 
ground CPs, each a collection of 9 co-located 64x64 image 
chips from Landsat Thematic Mapper (TM) images of an 
identifiable ground feature located to 30m. Each CP was 
selected for seasonally invariant features and then mapped 
into terrain-corrected imagery. A ray-casting algorithm is 
used to warp each chip to the appropriate geometry for the 
appropriate MISR camera. Chip matching is done to sub- 
pixel accuracy in translation, potentially to 1/8 pixel, by 
least squares optimization [1]. Cloud masking is used and 
outliers are rejected. 

MODIS is a third instrument on the Terra satellite. 
Wolfe et al. 2002 [18] describe the geolocaton system (see 
also [31]). MODIS has 36 spectral bands that sample at 
lkm, 500m and 250m The MODIS Land Science Team 
set a requirement of geolocation of 150m with an opera- 
tional goal of 50m Registration is done to a database of 
605 land CPs known to 15m in three dimensions. For each 
CP, 24km 2 chips of 30m sampling were constructed from 
Landsat TM bands 3 and 4 taken from cloud-free images 
predominately along coastlines and waterways. The higher- 
resolution TM chips are resampled to MODIS resolution us- 
ing the MODIS point spread function and nominal MODIS 
position information. Area-based correlation is used for 
matching, with a 25m sampling interval to get subpixel ac- 
curacy and a threshold of 0.6. Matches are used to update a 
camera geometric model that accounts for internal and ex- 
ternal calibration and gives per pixel geometric corrections. 


HRS, or High Resolution Stereoscopic with sampling at 
2.5m, is an instrument on the SPOT 5 satellite launched in 
2002. Baillarin et al. 2005 [3] describe the ANDORRE 
system with the TARIFA core for image fusion and recti- 
fication. HRS requirements are 90% of points within 16m 
circular accuracy and 10m elevation accuracy. ANDORRE 
uses an extensive database of orthoimagery tiles integrated 
with DEM data to generate, using ray tracing, a simulated 
image for matching. Matching is done using multresolution 
search with the number of levels set to keep a 5x5 pixel size 
search window. CPs are automatically found, matched by 
correlation and used to calibrate a parametric model. Ge- 
ometric outliers and CPs with correlation coefficient below 
0.80 are rejected. In the case of an inadequate number of tie 
points the correlation threshold is lowered to 0.70 and the 
number of points increased (’’densification”); a subscene of 
the image still without a sufficient number is rejected. The 
authors cite a final circular error of 2m for 88% of the points 
in 100 test images. Problems occur due to seasonal changes, 
low intensity images and homogenous subscenes. 

ETM+ is an instrument on the Landsat-7 satellite 
launched in 1999. Lee et al. [19] describe the extensive on- 
going set of assessment and calibration activities for ETM+ 
including image registration for geolocation. ETM+ re- 
quirements are for 250m absolute geodetic accuracy, 0.28 
pixels band-to-band resolution and 0.4 pixels temporal reg- 
istration. The three requirements lead to multiple registra- 
tion approaches for calibration and assessment. Geodetic 
accuracy is updated by correlating systematically corrected 
panchromatic band regions against a database of CP image 
chips extracted from USGS digital orthophoto quadrangles 
with vertical information from USGS DEMs. About 75- 
100 CP chips are used for assessing accuracy. Band-to-band 
registration is used to update a parameter file with corrected 
off-sets, so selected images in arid regions with little vege- 
tation and significant high spatial frequency content can im- 
prove cross-band correlation. Subpixel registration is done 
by second-order fit to the correlation surface. Temporal res- 
olution is assessed by correlating a reference image with 
a precision and terrain corrected panchromatic band of the 
test image. Success is reported for absolute geodetic accu- 




racy as 99.9% of test CPs within 250m and a 54m RMS 
error, while for band-to-band it is approximately 1/4 to 1/2 
of the operational requirement. Problems have been seen in 
decay of satellite attitude data due to aging gyros, and band- 
to-band calibration due to post-launch and seasonal operat- 
ing temperatures. Band-to-band correlation of band 6 (in- 
frared) has been problematic due to radiometric differences 
including seasonal temperature variation in arid regions. 

VEGETATION is an instrument on the SPOT series 
of satellites, operational since SPOT4 in 1999. Sylander 
et al. 2000 [24] describe the geolocation system. The re- 
quirements are 1km in absolute location, 500m in multi- 
temporal registration with an operational objective of 300 
m, and multispectral registration of 300m Initial work in- 
dicated that existing coastline databases were not accurate 
enough, and reliance on systematic correction was not ad- 
equate so a database of approximately 3500 CPs was built 
from VEGETATION images. Each CP has four image chips 
from different seasons and orientations. Matching using 
correlation is done under the control of human operators 
who insure there are 10 matched points per orbit to com- 
pute satellite position corrections. Absolute location was 
measured to 300m RMS, multitemporal to 325m in RMS, 
and multispectral was reported as 0. 1 1 pixels, which at that 
time was slightly outside the requirements. Problems occur 
due to seasonal changes, images with inadequate distribu- 
tion of CPs, rejection of images such as islands where CPs 
cannot be uniformly distributed, and CPs that may have ter- 
rain features poorly suited for correlation. 

From this survey we can see several common features 
and needs. The systems all share the use of pre-constructed 
databases, either of CPs or vector coastlines; the use of nor- 
malized intensity (or vector-edge) correlation as the similar- 
ity measure between features; DEM-correction of features 
before correlation; matching in local regions rather than 
globally; the use of translation since it dominates in small 
regions; cloud masking or thresholds to eliminate cloudy re- 
gions. Most systems use subpixel estimation but vary in the 
method used to compute it. The systems also vary in how 
local matches are used in precision correction - whether 
they are combined to estimate a global image transforma- 
tion or an update to the geometric camera model for sys- 
tematic correction. High-resolution sensors, on the order 
of 1 m, face specific challenges in accounting for terrain 
elevation effects and temporal changes caused by humans 
and other Earth processes. The problems reported by the 
groups show some differences but add up to the follow- 
ing picture of current registration issues: effectiveness of 
normalized correlation in cross-band registration; adaption 
to thermal changes in satellite geometry and minor prob- 
lems in orbit data; uniform distribution of CPs adequate for 
further numerical computations; CPs that are poorly suited 
for correlation for a number of reasons, including seasonal 


changes in temperature or vegetation or homogenous terrain 
features, and properly modeling of satellite position varia- 
tions. 

2.2. End user application requirements 

In contrast to the operational teams, end users do not al- 
ways have the same resources such as intimate knowledge 
of the sensor and data processing, the full set of sensor im- 
agery, the ability or need to maintain a large database of 
CPs, access to navigation modeling and other correction 
data, or statistical models of geolocation error. In addition, 
they may need to register heterogeneous data sources such 
as maps and older aerial images for which the supporting 
information is not available or complete. But, they do bring 
to image registration tasks knowledge of their application 
needs, most specifically accuracy needs and application- 
specific models. 

Accuracy needs of image registration for change detec- 
tion has been one of the most studied application needs [27, 
28, 6]. Townshend et al. 1992 [27] found that for four re- 
gions studied in Landsat TM imagery, to keep NDVI (Nor- 
malized Difference Vegetation Index) pixel-based change 
estimations down to an error of 10% required subpixel reg- 
istration of 0.2 pixels. However, for three more homoge- 
nous, arid regions they found that registrations of 0.5 to 1.0 
pixels were adequate for 10% error. Dai and Khorram 1998 
[10] report similar results with about 0.2 pixels registration 
required for 10% error on average in four test sites, with a 
slight but similar dependency on the spatial frequency con- 
tent of the imagery. Other confounding issues considered 
for misregistration impact on change detection has been the 
number of classes and region heterogeneity (Verbyla and 
Boles 2000 [28]), as well as change detection resolution and 
more through measures of region heterogeneity in high res- 
olution imagery (Wang and Ellis 2005 [30]). Authors have 
also considered how to adapt to misregistration errors by 
estimating them or lowering the resolution of change detec- 
tion [28, 30, 6]. 

Knowledge and use of appropriate transformation mod- 
els can assist in improving registration accuracy. Toutin 
2004 [26] extensively reviews transformation models used 
for satellite imagery ortho-rectification. Physical mod- 
els that take into account satellite geometry and the sen- 
sor imaging model, such as those used by the operational 
groups, can be the most accurate but are not always avail- 
able. Empirical models, such low order affine or RST, 
higher order polynomial, or piecewise transformations, do 
not require detailed sensor and satellite models but may 
require more CPs. Three-dimensional rational functions 
(RFs), ratios of polynomials, can approach the accuracy 
of physical models and are distributed by some imagery 
providers in meta-data but are not commonly used in au- 
tomatic registration algorithms. 



There are good studies that give extensive analyses of 
end user image registration experience and accuracy. Wang 
and Ellis 2005 [29] give such a detailed description and 
analysis for the registration by manual CPs of IKONOS and 
historical aerial photographs in China, detailing the acqui- 
sition by GPS of precise ground CPs for the imagery, study- 
ing the impact of various factors including terrain elevation 
variation and number of CPs, and using independent check 
points (ICPs) to measure accuracy. Armston et al. 2002 [2] 
gives a similar analysis of geometric correction of Landsat 
imagery over Australia, giving comparisons of physical and 
RF transformation models (the latter produced higher error) 
and looking at the spatial distribution of errors for control 
and check points. They give the rules used for selection of 
CPs, which included syntactic properties (choose high con- 
trast in two orthogonal directions, within 5x5 window) and 
semantic properties (avoid cleared land and bodies of water 
and choose bare ground among vegetation.) 

The assessment of transformations produced by manual 
CP selection is often judged by precision rather than for- 
mally by accuracy, as measures like RMS error are used that 
evaluate tightness of variance of the residuals for CPs Brov- 
elli et al. 2006 [4], This is often done with the residual under 
the computed transformation to a separate set of indepen- 
dent check points using holdout validation, or with leave- 
one-out cross-validation (LOOCV) which recomputes the 
transformation with N — 1 subsets and computes the resid- 
ual on the remaining point. Spatial and orientation distribu- 
tions of residuals have also been considered (Armston et al. 
2002 [2], Buiten and van Putten 1997 [7]). In general, accu- 
racy increases with the number of CPs up to a diminishing 
return [26, 29]. 

From this brief survey of end user requirements for reg- 
istration there can be seen connections with the experience 
of the operational groups. Homogenous regions that cause 
problems for image registration correlation may have com- 
pensating effects in reducing the accuracy requirements in 
applications like change detection; it may be possible to de- 
velop models for end users that, given the application re- 
quirements and the images to be processed, will specify up- 
dated and realistic accuracy requirements. 

3. Studies for evaluation of image registration 

Work of the NASA image registration group has empha- 
sized the study and evaluation of automatic image regis- 
tration algorithms for several purposes: precision correc- 
tion of satellite imagery, band-to-band calibration, dimen- 
sion and data reduction for ease of transmission, temporal 
multi-sensor fusion and on-board processing. Despite the 
wide variety of algorithms available for image registration, 
no commercial software seems to respond to the needs of 
Earth and space data registration. Our objective is to carry 
out systematic studies to support image registration users 


in selecting appropriate techniques for a remote sensing ap- 
plication based on accuracy and suitability for that applica- 
tion. We carry this out by surveying, designing and devel- 
oping different components of the registration process so 
we can evaluate their performance on well-chosen multiple 
source data, provide quantitative inter-comparison and to 
build an operational image registration toolbox. Following 
these goals, we performed several sets of experiments, and 
according to the observations described in the previous sec- 
tions, the challenges that were addressed in our experiments 
were concerned with different aspects of the same features 
in various temporal, various spatial resolutions and various 
spectral data, and obtaining subpixel accuracy. 

Assessing an image registration algorithm for subpixel 
accuracy on remote sensing data presents some difficulty 
since ground truth is not available. Interleaving two im- 
ages for visual assessment can detect gross mismatches and 
global alignment but is difficult to extend to quantitative 
subpixel evaluation. Manual registration can be used to cal- 
culate the unknown transformation but it is uncertain if it 
accurate enough to test subpixel accuracy on small regions. 
A second approach is to generate synthetic image pairs by 
matching one image against a transformed and resampled 
version of itself with or without added noise. To avoid some 
resampling issues this can be done by using high resolution 
imagery and downsampling to a lower resolution using an 
appropriate point spread function to generate both images in 
a pair. However, while useful this approach is limited in re- 
alistically modeling noise, temporal scene changes or cross- 
sensor issues. A third approach is to use circular registration 
results on natural imagery when three or more overlapping 
images are available. In this case the transformations should 
compose to yield the identity- for three images registered 
pairwise by 7j, T 2 and T 3 , the composition Tj o T 2 o T :i 
should be close to the identity transformation [34]. 

In all our experiments, we assume that the data have al- 
ready been corrected according to a navigation model and 
we can assume this systematic correction is accurate within 
a few or a few tens of pixels. Our precision-correction algo- 
rithms utilize selected image features or CPs to refine this 
geolocation accuracy within one pixel or a subpixel at the 
CPs. We first performed preliminary experiments summa- 
rized in [16] looking at different components individually 
[23, 33] and then developed a modular image registration 
framework [17] that enables systematic studies comparing 
these components and to test new components in a rigorous 
fashion. 

The first series of experiments investigated the use of 
different features with exhaustive correlation search: gray 
levels, edges, orthonormal/orthogonal wavelets (such as 
Haar and Daubechies) and over-complete decompositions 
(such as the Simoncelli steerable pyramid). The results 
showed that, as expected, edge or edge-like features (such 



as wavelets) were more robust to noise and local intensity 

variations than pure intensity values. In comparing edges 
and wavelets, a full exhaustive search using edge features 
proved to be potentially more accurate but was also time- 
prohibitive compared to an approach based on features ex- 
tracted from an orthogonal wavelet. 

The next series of experiments studied the potential lim- 
its of using wavelets, and particularly the effects of image 
translation on wavelet-based image registration [23], It is 
well-known that wavelet representations are not translation- 
invariant since when a separable orthogonal wavelet trans- 
form is used, information about the signal changes within 
or across sub-bands. Within these restrictions we quantita- 
tively assessed the usefulness of orthogonal wavelets. The 
main result was that the normalized correlation coefficients 
of low-pass orthogonal wavelet subbands are essentially in- 
sensitive to translations for features larger than twice the 
wavelet filter size. The third-level low-pass subbands pro- 
duce a correlation peak that varies with translation from 0.7 
to 1.0 with an average of 0.9. On the other hand, high- 
pass subbands are more sensitive to translation, although 
this sensitivity is limited. 

A follow-on series of experiments focused on evaluating 
several wavelet-type pyramids that may be utilized both for 
invariant feature extraction and for representing images at 
multiple spatial resolutions to accelerate registration. Us- 
ing an optimization-based scheme, Daubechies orthogonal 
wavelets, spline wavelets and the Simoncelli steerable Pyra- 
mid were considered. We found that the band-pass wavelets 
obtained from the Steerable Pyramid due to Simoncelli per- 
form best in terms of accuracy and consistency, while the 
low-pass wavelets obtained from the same pyramid give the 
best results in terms of the radius of convergence. Although 
not as reliable, spline wavelets show the best accuracy when 
convergence occurs. 

A more general set of experiments investigated combina- 
tions of the major image registration components: features, 
similarity metrics and search strategy. We built a framework 
to selectively combine the components in order to reach op- 
timum registration on a given type of data and under given 
circumstances. Another use of the framework is to test each 
element of registration independently (so we know the edge 
detection implementation, for example, is the same in all 
experiments.) The purpose of this framework is triple-fold: 
(1) assess various combinations of components as a func- 
tion of the application, (2) assess a new registration com- 
ponent compared to other known ones, and (3) serve as a 
registration tool where a user can select components as a 
function of the application at hand, the available computa- 
tional resources and the required accuracy. 

In our current framework: 

® features can be either gray levels, Low-Pass fea- 
tures from Simoncelli steerable filters decomposition 


or from a Spline decomposition, or Simoncelli Band- 
Pass features, similarity metrics can be either cross- 
correlation, the L2 Norm, Mutual Information or an 
Hausdorff distance, 

9 matching strategies are either based on a Fast Fourier 
Correlation, three different types of optimization - 
pure gradient Descent [1 1], a Marquard-Levenberg ap- 
proach [25] or a stocchastic gradient approach [9], and 
a Robust Feature Matching approach [21], 

Studies using this framework are described in [17], and 
some of the first results are the following: 

9 Fast Fourier correlation is very fast but limited to trans- 
formations with scales close to 1 (in the range [0.95, 
1.2]) and images containing very small amounts of 
noise. 

9 When using a Marquart-Levenberg approach, features 
such as Simoncelli Band-Pass are more accurate and 
more robust to noise, with errors in the range of [0.032, 
0.25] pixel. At the same time, Simoncelli-Low Pass 
1 features are less sensitive to the initial guess than 
Band-Pass features. 

9 An approach using a stochastic gradient approach and 
a Mutual Information metrics is overall the most robust 
to initial conditions and the most accurate with errors 
in the range of [0.12, 0.18] pixel. 

Figure 2 illustrates the types of results we obtained in 
one of these studies, investigating the sensitivity of com- 
ponents to initial conditions. In this experiment compo- 
nents of the framework were tested using synthetic data as 
well as Landsat multi-temporal data and IKONOS, Land- 
sat, MODIS and SeaWIFS data over four well-chosen EOS 
Land Validation Core Sites. Synthetic images were created 
using a controlled process [33] designed to emulate real 
data. Starting from a given source image, three types of 
modifications are applied in various combinations to pro- 
duce synthetic test data: (1) geometric warping, (2) radio- 
metric variations and (3) addition of noise. Figure 2 shows 
the results of the Marquart-Levenberg optimization strat- 
egy combined with a Mutual Information metrics and three 
different feature choices, low-pass Spline-wavelet features, 
Simoncelli band-pass features and Simoncelli low-pass fea- 
tures, when applied to synthetic data. Shifts are varied from 
0 to 8 pixels, rotations are varied from 0 to 8 degrees and 
scales are varied from -0.7 to 1.4. The white areas depict 
the regions of convergence of the algorithms with an error 
less than 1 pixel. This particular experiment demonstrates 
how Simoncelli Band-Pass features are more sensitive to the 
initial guess than Splines or Simoncelli Low-Pass features, 
while these last features exhibit the largest area of conver- 
gence. 
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Figure 2. Sensitivity to initial conditions 


4. Conclusion 

In order to better understand requirements for the devel- 
opment and evaluation of image registration algorithms, this 
article has examined selected research issues by surveying 
the experience of operational satellite teams, application- 
specific requirements for Earth science, and our framework 
and experiments in the evaluation of image registration al- 
gorithms. From the experience and current practice of oper- 
ational groups, as well as documented requirements for end 
user applications, we can see the importance of subpixel 
registration for image subregions with increasing need to 
perform it robustly between sensors, particularly with the 
advent of very high resolution and hyperspectral imagery 
and more demanding requirements for applications. We 
also see the need for image registration techniques that are 
tailored to the needs of remote sensing applications with 
specific attention to sensor-related geometric transforma- 
tions and knowledge of accuracy requirements and image 
content. 
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